Positionally Informative Remote Display Selection Interface

ABSTRACT

A user interface presented on a local display eases user discrimination between multiple remote displays connected to a remotely controlled computer, thereby easing selection of remote display(s) for remote viewing on the local display. The user interface can incorporate information indicating physical positional orientations of remote displays relative to each other, such as whether one remote display is physically located to the left of another remote display. The icons can additionally illustrate sizes of the remote displays relative to each other. Each remote display can be represented within a particular user interface icon as a rectangle whose size and position relative to the other remote display(s) reflect that remote display&#39;s actual physical size and position relative to those other remote display(s). Different icons can show different subsets of active and inactive remote monitors, and can be selected to activate different subsets of remote monitors for presentation on the local display.

BACKGROUND

The present disclosure relates generally to user interfaces, and inparticular to a user interface that enables the user selection of remotedisplays for viewing at a local display.

Computers and other electronic devices are capable of communicating witheach other over networks such as local area networks, wide areanetworks, and the Internet. Approaches have been devised for allowingone computer to remotely control another computer over a network. AppleRemote Desktop, initially released by Apple Inc. on Mar. 14, 2002, is anexample of a software application that enables such remote control.Further details about a recent release of Apple Remote Desktop may befound in “Apple Remote Desktop Administrator Guide Version 3.3,” whichis incorporated by reference herein. In the discussion below, thecontrolling computer, which is located in the user's immediate presence,and to which the user has physical access, is referred to as the “localcomputer.” The remotely controlled computer, which is not necessarilylocated in the user's immediate presence, and to which the user does notnecessarily have physical access, is referred to below as the “remotecomputer.”

Often, in a remote control scenario, both the local computer and theremote computer will have directly connected displays or monitors, suchas active-matrix liquid crystal displays. A display may be directlyconnected to a computer via a High Definition Multimedia Interface(HDMI), a Video Graphics Array (VGA) connector, a Digital VisualInterface (DVI), or some other physical data communication port.Alternatively, a display can be communicatively coupled to the computerusing wireless technology. Additionally, using the “AirPlay” feature inMac OS X 10.8, a user can mirror a display to some other display. Suchtechnology can potentially be used to create a second display for acomputer rather than just mirroring the contents of a display.Furthermore, in the case of a computer that is only used for remoteaccess, there might not be actual hardware displays connected to thatcomputer at all; under such circumstances, the computer can createvirtual displays of any given width and height. On a Macintosh computer,one can set the logical arrangement of these virtual displays. Asdiscussed herein, “displays” includes both physical and virtualdisplays. Each computer will send, to the display that is directly orotherwise connected to that computer, signals that the display then usesto render an image that will be visually shown on the display. In thediscussion below, any display that is directly or otherwise connected tothe local computer is referred to as a “local display.” Any display thatis otherwise directly connected to the remote computer is referred tobelow as “remote display.”

In a remote control scenario, the remote computer will also send, overthe network to the local computer, data indicative of the image that iscurrently being shown on the remote display. In response to receivingthis data, the local computer sends, to the local display, signals thatrepresent this data. The local display then uses these signals to renderan image that is visually shown on the local display. Thus, the localdisplay may show images that are currently being shown on the remotedisplay, better enabling the user of the local computer to remotelycontrol the remote computer.

Under some circumstances, multiple remote displays might be directlyconnected to the remote computer. Under such circumstances, the localcomputer may execute software that presents, on the local display, auser interface that enables the user of the local computer to selectwhich one(s) of the remote displays are to have their currentlypresented images shown on the local display. The user interface mayidentify each remote display by a number, such as “Display 1” and“Display 2.” The user interface also may indicate, for each display,what the current resolution settings for that display are.Unfortunately, the user of the local computer often cannot determine,based only on this meager information presented via the user interface,which remote display is currently showing the image that the user wouldlike to view on the local computer. Even if the user has been to theremote site previously, the user might not know which of the multipleremote displays is being referred to in the user interface as “Display1,” and which of the multiple remote displays is being referred to inthe user interface as “Display 2.” Consequently, the user typicallyendures the aggravation of selecting the desired remote display througha trial-and-error process, in which the user selects each remote monitorin turn until the user sees the desired image on the local display.

SUMMARY

Certain embodiments of the present invention can present, on a localdisplay, a user interface that makes it easier for a user of a localcomputer to discriminate between the identities of multiple remotedisplays directly connected to a remote computer, so that the user canmore easily select the remote display(s) whose images the user wants tosee shown on the local display. In certain embodiments of the invention,positional orientation information, which indicates the physicalpositional orientations of the remote monitors relative to each other atthe remote site, can be incorporated into the user interface. Forexample, in certain embodiments of the invention, a user interface caninclude icons that illustrate a first remote display being to the leftof a second remote display, indicating to the user that the first remotedisplay is physically located to the left of the second display at theremote site. In certain embodiments of the invention, the icons canadditionally illustrate sizes of the remote displays relative to eachother. For example, in certain embodiments of the invention, if a firstremote display is twice as large as a second remote display, then theuser interface can include icons that illustrate the first remotedisplay being twice as large as a second remote display. In certainembodiments, each remote display can be represented within a particularuser interface icon as a rectangle whose size and position relative tothe other remote display(s) reflect that remote display's actualphysical size and position relative to those other remote display(s).

In certain embodiments of the invention, the user interface discussedabove includes a series of icons, each illustrating the positionalorientations of the remote displays relative to each other, and eachrepresentative of a different subset of those remote monitors currentlybeing active for viewing on the local display. For example, in certainembodiments of the invention, a first icon can illustrate a left remotedisplay being active concurrently with a right remote display beinginactive, while a second icon can illustrate a left remote display beinginactive concurrently with a right remote display being active, while athird icon can illustrate both the left and right remote displays beingactive concurrently. In such embodiments of the invention, a user'sselection (e.g., via mouse click) of a particular one of the icons cancause (a) the activation of the remote displays illustrated in thatparticular icon as being active, simultaneous with (b) the deactivationof the remote displays illustrated in that particular icon as beinginactive (if any). In certain embodiments of the invention, in responseto the activation of one or more remote displays through user selectionof a particular icon, images currently being presented on thoseactivated remote displays are presented on the local display.Conversely, in certain embodiments of the invention, in response to thedeactivation of one or more remote displays through user selection of aparticular icon, images currently being presented on those deactivatedremote displays cease to be presented on the local display.

The following detailed description together with the accompanyingdrawings will provide a better understanding of the nature andadvantages of the present invention.

BRIEF DESCRIPTION

FIG. 1 is a block diagram of a computer system according to anembodiment of the present invention.

FIG. 2 is block diagram illustrating a system in which multiple remotedisplays can be remotely controlled and viewed on a local display,according to an embodiment of the invention.

FIG. 3 is a block diagram illustrating a user interface that providespositional orientation information regarding remote displays that can beselected for viewing on a local display, according to an embodiment ofthe invention.

FIG. 4 is a block diagram that illustrates a local display showing tworemote display-representing frames and concurrently, according to anembodiment of the invention.

FIG. 5 is a block diagram illustrating an alternative user interfacethat provides positional orientation information regarding remotedisplays that can be selected for viewing on a local display, accordingto an embodiment of the invention.

FIG. 6 is a diagram illustrating a sequence of communications between alocal computer and a remote computer via a network in order to conveyinformation sufficient for the local computer to cause a local displayto present a positionally informative user interface, according to anembodiment of the invention.

DETAILED DESCRIPTION

FIG. 1 illustrates a computer system 100 according to an embodiment ofthe present invention. Computer system 100 can be implemented as any ofvarious computing devices, including, e.g., a desktop or laptopcomputer, tablet computer, smart phone, personal data assistant (PDA),or any other type of computing device, not limited to any particularform factor. Computer system 100 can include processing unit(s) 105,storage subsystem 110, input devices 120, display 125, network interface135, and bus 140.

Processing unit(s) 105 can include a single processor, which can haveone or more cores, or multiple processors. In some embodiments,processing unit(s) 105 can include a general-purpose primary processoras well as one or more special-purpose co-processors such as graphicsprocessors, digital signal processors, or the like. In some embodiments,some or all processing units 105 can be implemented using customizedcircuits, such as application specific integrated circuits (ASICs) orfield programmable gate arrays (FPGAs). In some embodiments, suchintegrated circuits execute instructions that are stored on the circuititself. In other embodiments, processing unit(s) 105 can executeinstructions stored in storage subsystem 110.

Storage subsystem 110 can include various memory units such as a systemmemory, a read-only memory (ROM), and a permanent storage device. TheROM can store static data and instructions that are needed by processingunit(s) 105 and other modules of electronic device 100. The permanentstorage device can be a read-and-write memory device. This permanentstorage device can be a non-volatile memory unit that storesinstructions and data even when computer system 100 is powered down.Some embodiments of the invention can use a mass-storage device (such asa magnetic or optical disk or flash memory) as a permanent storagedevice. Other embodiments can use a removable storage device (e.g., afloppy disk, a flash drive) as a permanent storage device. The systemmemory can be a read-and-write memory device or a volatileread-and-write memory, such as dynamic random access memory. The systemmemory can store some or all of the instructions and data that theprocessor needs at runtime.

Storage subsystem 110 can include any combination of computer readablestorage media including semiconductor memory chips of various types(DRAM, SRAM, SDRAM, flash memory, programmable read-only memory) and soon. Magnetic and/or optical disks can also be used. In some embodiments,storage subsystem 110 can include removable storage media that can bereadable and/or writeable; examples of such media include compact disc(CD), read-only digital versatile disc (e.g., DVD-ROM, dual-layerDVD-ROM), read-only and recordable Blu-Ray® disks, ultra density opticaldisks, flash memory cards (e.g., SD cards, mini-SD cards, micro-SDcards, etc.), magnetic “floppy” disks, and so on. The computer readablestorage media do not include carrier waves and transitory electronicsignals passing wirelessly or over wired connections.

In some embodiments, storage subsystem 110 can store one or moresoftware programs to be executed by processing unit(s) 105, such as aremote desktop viewer 145. “Software” refers generally to sequences ofinstructions that, when executed by processing unit(s) 105 causecomputer system 100 to perform various operations, thus defining one ormore specific machine implementations that execute and perform theoperations of the software programs. The instructions can be stored asfirmware residing in read-only memory and/or applications stored inmagnetic storage that can be read into memory for processing by aprocessor. Software can be implemented as a single program or acollection of separate programs or program modules that interact asdesired. Programs and/or data can be stored in non-volatile storage andcopied in whole or in part to volatile working memory during programexecution. From storage subsystem 110, processing unit(s) 105 canretrieves program instructions to execute and data to process in orderto execute various operations described herein.

A user interface can be provided by one or more user input devices 120,display device 125, and/or and one or more other user output devices(not shown). Input devices 120 can include any device via which a usercan provide signals to computing system 100; computing system 100 caninterpret the signals as indicative of particular user requests orinformation. In various embodiments, input devices 120 can include anyor all of a keyboard, touch pad, touch screen, mouse or other pointingdevice, scroll wheel, click wheel, dial, button, switch, keypad,microphone, and so on.

Display 125 can display images generated by electronic device 100 andcan include various image generation technologies, e.g., a cathode raytube (CRT), liquid crystal display (LCD), light-emitting diode (LED)including organic light-emitting diodes (OLED), projection system, orthe like, together with supporting electronics (e.g., digital-to-analogor analog-to-digital converters, signal processors, or the like). Someembodiments can include a device such as a touchscreen that function asboth input and output device. In some embodiments, other user outputdevices can be provided in addition to or instead of display 125.Examples include indicator lights, speakers, tactile “display” devices,printers, and so on.

In some embodiments, the user interface can provide a graphical userinterface, in which visible image elements in certain areas of display125 are defined as active elements or control elements that the user canselect using user input devices 120. For example, the user canmanipulate a user input device to position an on-screen cursor orpointer over the control element, then click a button to indicate theselection. Alternatively, the user can touch the control element (e.g.,with a finger or stylus) on a touchscreen device. In some embodiments,the user can speak one or more words associated with the control element(the word can be, e.g., a label on the element or a function associatedwith the element). In some embodiments, user gestures on atouch-sensitive device can be recognized and interpreted as inputcommands; these gestures can be but need not be associated with anyparticular array in display 125. Other user interfaces can also beimplemented.

Network interface 135 can provide voice and/or data communicationcapability for electronic device 100. In some embodiments, networkinterface 135 can include radio frequency (RF) transceiver componentsfor accessing wireless voice and/or data networks (e.g., using cellulartelephone technology, advanced data network technology such as 3G, 4G orEDGE, WiFi (IEEE 802.11 family standards, or other mobile communicationtechnologies, or any combination thereof), GPS receiver components,and/or other components. In some embodiments, network interface 135 canprovide wired network connectivity (e.g., Ethernet) in addition to orinstead of a wireless interface. Network interface 135 can beimplemented using a combination of hardware (e.g., antennas,modulators/demodulators, encoders/decoders, and other analog and/ordigital signal processing circuits) and software components.

Bus 140 can include various system, peripheral, and chipset buses thatcommunicatively connect the numerous internal devices of electronicdevice 100. For example, bus 140 can communicatively couple processingunit(s) 105 with storage subsystem 110. Bus 140 also connects to inputdevices 120 and display 125. Bus 140 also couples electronic device 100to a network through network interface 135. In this manner, electronicdevice 100 can be a part of a network of multiple computer systems(e.g., a local area network (LAN), a wide area network (WAN), anIntranet, or a network of networks, such as the Internet. Bus 140further enables communication between electronic device 100 and anaccessory via accessory interface 155. Any or all components ofelectronic device 100 can be used in conjunction with the invention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in acomputer readable storage medium. Many of the features described in thisspecification can be implemented as processes that are specified as aset of program instructions encoded on a computer readable storagemedium. When these program instructions are executed by one or moreprocessing units, they cause the processing unit(s) to perform variousoperation indicated in the program instructions. Examples of programinstructions or computer code include machine code, such as is producedby a compiler, and files including higher-level code that are executedby a computer, an electronic component, or a microprocessor using aninterpreter.

Through suitable programming, processing unit(s) 105 can provide variousfunctionality for electronic device 100. For example, processing unit(s)105 can execute remote desktop viewer 145. In some embodiments, remotedesktop viewer 145 is a software-based process that can cause imagescurrently being presented on one or more remote displays to be shown ona local display such as display 125. In some embodiments, remote desktopviewer 145 presents, on local display 125, a user interface thatindicates positional orientations of multiple remote displays relativeto each other.

It will be appreciated that computer system 100 is illustrative and thatvariations and modifications are possible. Computer system 100 can haveother capabilities not specifically described here (e.g., mobile phone,global positioning system (GPS), power management, one or more cameras,various connection ports for connecting external devices or accessories,etc.). Further, while computer system 100 is described with reference toparticular blocks, it is to be understood that these blocks are definedfor convenience of description and are not intended to imply aparticular physical arrangement of component parts. Further, the blocksneed not correspond to physically distinct components. Blocks can beconfigured to perform various operations, e.g., by programming aprocessor or providing appropriate control circuitry, and various blocksmight or might not be reconfigurable depending on how the initialconfiguration is obtained. Embodiments of the present invention can berealized in a variety of apparatus including electronic devicesimplemented using any combination of circuitry and software.

Multiple Remote Display Control System

FIG. 2 is a block diagram illustrating a system 200 in which multipleremote displays can be remotely controlled and viewed on a localdisplay, according to an embodiment of the present invention. In theembodiment illustrated, system 200 includes a remote computer 202 and alocal computer 204. Remote computer 202 can communicate with localcomputer 204 via a network 210, to which both remote computer 202 andlocal computer 204 can be communicatively connected via wired orwireless connections. Network 210 can be or can include a local areanetwork, a wide area network, and/or the Internet. Displays 206A-N canbe communicatively and directly connected to remote computer 204 viainterfaces such as HDMI, VGA connectors, DVI, or other physical ports.Although multiple remote displays 206A-N are illustrated within system200, various different embodiments of the invention can include variousdifferent quantities of remote displays. In one embodiment of theinvention, there can be two remote displays. Displays 206A-N can beactive-matrix liquid crystal displays, cathode ray tube displays, or anyother type of display or monitor capable of rendering and showing imagesrepresented by signals that displays 206A-N can receive from remotecomputer 202. Remote computer 202 can send different signalsrepresenting different images to each of displays 206A-N concurrently,so that the images presented by each of displays 206A-N may differ fromeach other.

In one embodiment of the invention, a server application can execute onremote computer 202, while a client application can execute concurrentlyon local computer 204. These applications can communicate with eachother over network 210 using a request-response communication model.Through these inter-application communications, local computer 204 canrequest and receive, from remote computer 202, data representing imagesfrom one or more of remote displays 206A-N. Remote computer 202 can beconnected directly and communicatively with local display 208 using anyof the connectivity options discussed above relative to remote computer202 and remote displays 206A-N. In addition to an image that localcomputer 204 can generate and cause local display 208 to show, localcomputer 204 can additionally or alternatively cause local display 208to show images represented by any of the data that local computer 204receives from remote computer 202 over network 210. Thus, at any giventime, local computer 204 can cause local display 208 to display imagesthat are currently being shown on one or more of remote displays 206A-N.

In one embodiment of the invention, local computer 204 can cause localdisplay 208 to display, concurrently, multiple images from multipledifferent ones of remote displays 206A-N. For example, local display 208might present two separate frames concurrently, one of which contains animage that is being shown on remote display 206A, and the other of whichcontains an image that is being shown on remote display 206B. As theimages on remote displays 206A-N change over time, remote computer 202can send updated data over network 210 to local computer 204 in order toenable local computer 204 to update the corresponding images shown onlocal display 208.

In one embodiment of the invention, local computer 204 can generate auser interface that includes one or more icons that enable the user oflocal computer 204 to designate various different subsets of remotemonitors 206A-N to be active and inactive for remote control purposes.In one embodiment of the invention the user's selection of a particularone of these icons can (a) cause local computer 204 to cause localdisplay 208 to begin showing images from those of remote monitors 206A-Ndesignated to be active according to that particular icon andsimultaneously (b) cause local computer 204 to cause local display 208to cease showing images from those of remote monitors 206A-N designatedto be inactive according to that particular icon. An example of a userinterface with selectable icons is discussed in further detail below.

Positionally Informative User Interface

FIG. 3 is a block diagram illustrating a user interface 300 thatprovides positional orientation information regarding remote displaysthat can be selected for viewing on a local display, according to anembodiment of the invention. In the embodiment illustrated, userinterface 300 includes three user-selectable icons 302, 304, and 306.However, in alternative embodiments of the invention, a greater orlesser quantity of icons may be included in the user interface. Forexample, the quantity of icons may depend on the quantity of remotedisplays 206A-N; the user interface can contain an icon for each remotedisplay, plus one icon for all of the displays combined.

Icons 302-306 all indicate the positional orientations of two remotedisplays—e.g., remote displays 206A and 206B—relative to each other viarectangles depicted within those icons. Each icon depicts a separaterectangle for each remote display directly connected to the remotecomputer. Icons 302-306 each show that one remote display is physicallylocated to the left of the other remote display at the remote site, andalso that the remote display on the left is elevated slightly related tothe remote display on the right. This depiction can help the user toidentify the remote displays and distinguish between them, as thedepiction presents a view of the remote displays more similar to the onethat the user would see if the user were at the remote site instead ofat the local computer. If one remote display has different dimensionsthan the other remote display, then the shape and/or size of therectangles representing the remote displays within icons 302-306 can beadjusted accordingly to reflect these differing dimensions. Onerectangle may have a different width or height than the other rectangleif one remote display has a different width or height than the otherremote display. Although in one embodiment icons 302-306 reflectphysical arrangements of displays set by a user using a configurationutility, in alternatively embodiments of the invention, icons 302-306instead reflect a logical arrangement of displays set by such a userusing such a configuration utility, and such a logical arrangement mightor might not match the actual physical arrangement of the displays.Alternatively, in one embodiment of the invention, the displays can bevirtual displays having logical arrangement set by a user using aconfiguration utility.

In the embodiment of the invention illustrated in FIG. 3, each of icons302-306 represents a different subset of active remote displays. Shadedrectangles can indicate active remote displays in the icon, whileunshaded rectangles can indicate inactive remote displays in that icon.For example, icon 302 has a shaded left rectangle and an unshaded rightrectangle, indicating that if icon 302 is selected (e.g., via a mouseclick), then the left remote display will become active and the rightremote display will become inactive. In contrast, icon 304 has anunshaded left rectangle and a shaded right rectangle, indicating that ificon 304 is selected, then the left remote display will become inactiveand the right remote display will become active. Icon 306 has shadedleft and right rectangles, indicating that if icon 306 is selected, thenboth the left and right remote displays will become active concurrently.In one embodiment of the invention, the user selection of a particularicon from icons 302-306 causes the icon-indicated remote displays tobecome active and/or inactive. In one embodiment of the invention,contents of, or the images currently shown on, active remote displaysare also presented on local display 208, while such contents and imagesfrom inactive remote displays are not presented on local display 208. Inone embodiment of the invention, the user selection of a particular iconfrom icons 302-306 causes that particular icon to be highlighted, or tobe displayed in reverse video, or to be otherwise visually distinguishedfrom the other icons to indicate the current selection of thatparticular icon rather than any of the other icons. In one embodiment ofthe invention, a particular icon from icons 302-306 remains selected andvisually distinguished until either that particular icon is unselected(e.g., via a mouse click) or until a different icon from icons 302-306is selected. Thus, in one embodiment of the invention, at most only oneof icons 302-306 is currently selected and visually distinguished at anyparticular moment.

In one embodiment of the invention, in response to a user's selection ofa particular icon from icons 302-306, local computer 204 can cause localdisplay 208 to show a separate frame for each of the active remotedisplays made active by the selection of the particular icon. Each suchframe can contain the contents of, or an image currently being shown on,the active remote display corresponding to that frame. As the images onthose active remote displays change, the images shown within thecorresponding frames on local display 208 can change accordingly andconsistently, such that the frames essentially continuously depict theimages that are currently being shown on the corresponding active remotedisplays at any given moment. In one embodiment of the invention, thiscontinuous updating of the frame contents can be accomplished by remotecomputer 202 continuously sending, over network 210 to local computer204, data representing the signals that remote computer 202 is directingto those of remote displays 206A-N that are currently active.

In one embodiment of the invention, the frames discussed above arepositioned relative to each other in the same manner in which thecorresponding rectangles from the currently selected particular icon, oficons 302-306, are positioned relative to each other. In one embodimentof the invention, the frames discussed above have dimensions and sizesrelative to each other that are representative of the dimensions andsizes of the corresponding rectangles from the currently selectedparticular icon, of icons 302-306. Thus, for example, if icon 302 iscurrently selected, then local display 208 can present a single framethat has a size and dimensions commensurate with, or scaled based on,the size and dimensions of remote display 206A. For another example, ificon 304 is currently selected, then local display 208 can present asingle frame that has a size and dimensions commensurate with, or scaledbased on, the size and dimensions of remote display 206B. For yetanother example, if icon 306 is currently selected, then local display208 can present two frames simultaneously, each having sizes anddimensions based on the sizes and dimensions of the remote displays towhich those frames correspond. Additionally, each of these frames can bepresented at a location on local display 208 that is reflective of thephysical orientation and location of that frame's corresponding remotedisplay relative to the other remote display. For example, inasmuch asicons 302-306 depict the left remote display being to the left of andslightly elevated relative to the right remote display, the selection oficon 306 can cause local display 208 to present a frame for remotedisplay 206A that is to the left of and slightly elevated relative tothe frame that local display 208 concurrently presents for remotedisplay 206B.

FIG. 4 is a block diagram that illustrates a local display 400 showingtwo remote display-representing frames 402 and 404 concurrently,according to an embodiment of the invention. In the illustrated example,frame 402 corresponds to remote display 206A, while frame 404corresponds to remote display 206B. Because remote display 206A isphysically oriented to the left of and slightly elevated relative toremote display 206B, frame 402 is oriented to the left of and slightlyelevated relative to frame 404.

In one embodiment of the invention, in response to the user of localcomputer 204 causing (e.g., using a mouse device or other input device)a pointer to hover over a particular one of icons 302-306, localcomputer 204 can cause a “tooltip” or pop-up text to appear, at leasttemporarily, near to that particular icon. The tooltip can indicatevarious characteristics of the remote displays represented by therectangles in that icon. For example, the tooltip can indicate the pixeldimensions for each of the remote displays within the tooltip text.

FIG. 5 is a block diagram illustrating an alternative user interface 500that provides positional orientation information regarding remotedisplays that can be selected for viewing on a local display, accordingto an embodiment of the invention. As illustrated, user interface 500includes icons 502, 504, and 506. Each of icons (or menu options)502-506 represents a different subset of active ones of remote displays206A-N, similar to the manner in which icons 302-306 of FIG. 3 representsuch subsets. Icon 502 represents a subset in which remote displays 206Aand 206B are both active. Icon 504 represents a subset in which remotedisplay 206A is active and remote display 206B is inactive. Icon 506represents a subset in which remote display 206A is inactive and remotedisplay 206B is active. In an embodiment of the invention, userselection of a particular one of icons 502-506 can cause local display208 to present frames corresponding to the active displays representedby the particular icon, in a manner similar to that discussed above inconnection with FIG. 3.

Unlike icons 302-306 of FIG. 3, however, icons 502-506 of FIG. 5additionally contain text indicating names or identities of the remotedisplays whose local viewing will be activated by the user selection ofthose icons. Furthermore, each of icons 502-506 contains text indicatingthe pixel dimensions of the corresponding active remote display(s)depicted by shaded rectangles in that icon. Icon 502 contains textindicating that user selection of icon 502 will cause the activation ofboth remote displays, which together have combined pixel dimensions of2480×1050. Icon 504 contains text indicating that user selection of icon504 will cause the activation of the left remote display (and thedeactivation of the right remote display), which has pixel dimensions of1600×1000. Icon 506 contains text indicating that user selection of icon506 will cause the activation of the right remote display (and thedeactivation of the left remote display), which has pixel dimensions of1680×1050. Although one embodiment of the invention involves thepresentation of user-selectable icons containing text and a pictorialrepresentation of positional orientations of remote displays relative toeach other, as in FIG. 5, alternative embodiments of the inventioninstead can present user-selectable icons having a similar purpose andfunctionality without containing such pictorial representations. Forexample, in one alternative embodiment of the invention, user-selectabletextual menu options can textually convey positional orientationinformation of corresponding remote displays using words such as “left,”“right,” “top,” “bottom,” etc. For a more specific example, the displayreferred to as “Display 1” in FIG. 5 might be referred to instead as“left display,” while the display referred to as “Display 2” in FIG. 5might be referred to instead as “right display,” thereby conveying tothe user of local computer 204 that remote display 206A is physicallypositioned and oriented to the left of remote display 206B at the remotesite.

Network-Based Communications for User Interface Generation

FIG. 6 is a diagram illustrating a sequence 600 of communicationsbetween local computer 204 and remote computer 202 via network 210 inorder to convey information sufficient for local computer 204 to causelocal display 208 to present the positionally informative user interfacediscussed above, according to an embodiment of the invention. In step602, local computer 204 (potentially executing a client application thatcommunicates with a server application executing on remote computer 202)sends a log-on request to remote computer 202. The request can specify ausername and password, for example. In step 604, remote computer 202authenticates the user of local computer 204 based on the informationspecified in the request, and sends an authentication success message tolocal computer 204. In step 606, local computer 204 sends, to remotecomputer 202, a list of commands supported by local computer 204. Thesecommands can indicate, for example, the types of remote displaycharacteristics that local computer 204 is capable of understanding.Such types of characteristics may include, for example, positionalorientation information, pixel resolution information, pixel densityinformation, etc. Remote computer 202 can use this list of commands todetermine the types of information that remote computer 202 should sendto local computer 204 in subsequent interactions.

In one embodiment of the invention, remote computer 202 stores andmaintains information regarding the positional orientations and othercharacteristics of each of remote displays 206A-N. For example, in oneembodiment of the invention, the positional orientation of each ofremote displays 206A-N can be represented by numerical coordinates thatindicate physical positions of upper-left and lower-right corners ofthose remote displays in physical space at the remote site. Anadministrator of remote computer 202 can establish this positionalorientation information manually using a tool resident on remotecomputer 202. As is discussed above, although in one embodiment of theinvention the positional orientation information reflects the actualphysical positional orientation of physical displays, in alternativeembodiments of the invention, the administrator can use such a tool toestablish alternative positional orientation information that canreflect a logical arrangement of the displays that differs from theactual physical arrangement of the displays. Indeed, in one alternativeembodiment of the invention, the positional orientation information canrefer to virtual displays that do not physically exist, but arelogically established, dimensioned, and oriented by the administratorusing such a tool. In one embodiment of the invention, storedcharacteristics of remote displays 206A-N, including positionalorientation information, also can be modified at any time—for example,in concert with the physical addition or removal or movement of anyremote display to, from, or within remote displays 206A-N—and updated instorage using such a tool, or via automatic detection mechanisms. In oneembodiment of the invention, the updating of the stored characteristicscauses remote computer 202 to send updated data to local computer 204 inorder to enable local computer 204 to cause local display 208 to updateits user interface and displayed frames with the updatedcharacteristics. Thus, in one embodiment of the invention, if a remotedisplay is added or removed or physically moved, then this fact can bedetected, and local computer 204 can responsively generate updated iconsfor the user interface that local display 208 presents to the user.

In step 608, remote computer 202 sends, over network 210 to localcomputer 204, data indicating positional orientation information andother characteristics of remote displays 206A-N. In step 610, localcomputer 204 receives this data and, based at least in part on thisdata, generates a user interface including positionally informativeicons of the kind described above (e.g., in connection with FIG. 3and/or FIG. 5). Local computer 204 causes local display 208 to presentthe user interface. Local computer 204 can detect a user's selection ofa particular icon from the positionally informative icons included inthe presented user interface. Also in step 610, responsive to the userselection of a positionally informative icon from the user interface,local computer 204 sends, over network 210 to remote computer 202,information indicating which subset of remote displays 206A-N has beenselected to be active for remote viewing. In step 612, remote computer202 receives and stores this selection information. Thereafter, remotecomputer 202 can use this selection information to determine which ofthe signals that remote computer 202 sends to various ones of remotedisplays 206A-N also should be relayed across network 210 to localcomputer 202; in one embodiment of the invention, remote computer 202only sends, to local computer 202, image-representing data for those ofremote displays 206A-N that were designated as being active in step 610.

Embodiments of the present invention can be realized using anycombination of dedicated components and/or programmable processorsand/or other programmable devices. The various processes describedherein can be implemented on the same processor or different processorsin any combination. Where components are described as being configuredto perform certain operations, such configuration can be accomplished,e.g., by designing electronic circuits to perform the operation, byprogramming programmable electronic circuits (such as microprocessors)to perform the operation, or any combination thereof Further, while theembodiments described above can make reference to specific hardware andsoftware components, those skilled in the art will appreciate thatdifferent combinations of hardware and/or software components can alsobe used and that particular operations described as being implemented inhardware might also be implemented in software or vice versa.

Computer programs incorporating various features of the presentinvention can be encoded and stored on various computer readable storagemedia; suitable media include magnetic disk or tape, optical storagemedia such as compact disk (CD) or DVD (digital versatile disk), flashmemory, and other non-transitory media. Computer readable media encodedwith the program code can be packaged with a compatible electronicdevice, or the program code can be provided separately from electronicdevices (e.g., via Internet download or as a separately packagedcomputer-readable storage medium).

Thus, although the invention has been described with respect to specificembodiments, it will be appreciated that the invention is intended tocover all modifications and equivalents within the scope of thefollowing claims.

What is claimed is:
 1. A method for representing remote displays, themethod comprising: receiving, at a first device, information thatindicates at least a positional orientation of (1) at least a firstdisplay connected to a second device relative to (2) at least a seconddisplay connected to said second device; and responsive to receiving theinformation, presenting, at the first device, a representationindicative of the positional orientation of at least the first displayrelative to at least the second display.
 2. The method of claim 1,wherein the first device receives the information over a network fromthe second device, wherein the positional orientation includes both (a)a size and position of the first display and (b) a size and position ofthe second display, and wherein presenting the representation comprises:presenting, at the first device, a graphical indication of the size andposition of the first display relative to the size and position of thesecond display.
 3. The method of claim 1, wherein the first devicereceives the information over a network from the second device, whereinthe positional orientation includes both (a) a size and position of thefirst display and (b) a size and position of the second display, andwherein presenting the representation comprises: presenting, at thefirst device, a textual indication of the size and position of the firstdisplay relative to the size and position of the second display.
 4. Themethod of claim 1, wherein presenting the representation comprises:presenting, at the first device, a set of icons, each particular icon ofwhich indicates sizes and positions of the first and second displaysrelative to each other, and each particular icon of which is selectableby a user of the first device to select, from a set of contentcomprising both contents from the first display and contents from thesecond display, a different subset of the set of content to be displayedat the first device.
 5. The method of claim 1, wherein presenting therepresentation comprises presenting, at the first device, a set of iconsthat indicate sizes and positions of the first and second displaysrelative to each other, and further comprising: detecting, at the firstdevice, a user selection of a particular icon from the set of icons; andin response to the detecting, displaying, at the first device, at afirst position that is based at least in part on positional informationindicated by the positional orientation, a first frame having a sizethat is based at least in part on size information indicated by thepositional orientation for the first display; in response to thedetecting, displaying, at the first device, at a second position that isbased at least in part on positional information indicated by thepositional orientation, a second frame having a size that is based atleast in part on size information indicated by the positionalorientation for the second display; displaying contents from the firstdisplay in the first frame; and displaying contents from the seconddisplay in the second frame.
 6. The method of claim 1, whereinpresenting the representation comprises presenting, at the first device,a set of icons that indicate sizes and positions of the first and seconddisplays relative to each other, and further comprising: detecting, atthe first device, a user selection of a particular icon from the set oficons; and in response to the detecting, ceasing to display, at thefirst device, a first frame (a) having a size that is based at least inpart on size information indicated by the positional orientation for thefirst display and (b) containing contents from the first display; inresponse to the detecting, displaying, at the first device, a secondframe having a size that is based at least in part on size informationindicated by the positional orientation for the second display; anddisplaying contents from the second display in the second frame.
 7. Anon-transitory computer-readable storage medium storing instructionswhich, when executed by one or more processors, cause the one or moreprocessors to perform steps comprising: sending, over a network, to afirst device, first information indicating a first position of a firstdisplay relative to a second position of a second display; receiving,over the network, from the first device, second information indicating aselection, by a user of the first device, of a particular display thatis either the first display or the second display; and in response toreceiving the second information, sending, to the first device, thirdinformation that the first device uses to generate, on a third display,a visual representation of contents from the particular display.
 8. Thenon-transitory computer-readable storage medium of claim 7, wherein thestep of sending the first information comprises sending coordinates ofthe first display and sending coordinates of the second display.
 9. Thenon-transitory computer-readable storage medium of claim 7, wherein thestep of sending the first information comprises sending information thatthe first device uses to generate at least one icon that visuallyillustrates an orientation of (1) a currently selected one of the firstand second displays relative to (2) a currently non-selected one of thefirst and second displays.
 10. The non-transitory computer-readablestorage medium of claim 7, wherein the step of sending the firstinformation comprises sending information that the first device uses togenerate a user interface that indicates that the first display isphysically to the left of the second display.
 11. A non-transitorycomputer-readable storage medium storing instructions which, whenexecuted by one or more processors, cause the one or more processors toperform steps comprising: receiving, over a network, from a seconddevice, first information indicating that a first display connected tothe second device is physically to the left of a second displayconnected to the second device; and in response to receiving the firstinformation, generating a user interface element that (1) indicates thatthe first display is physically to the left of the second display and(2) indicates which of the first and second display's contents iscurrently being presented to a user of a first device; and presentingthe user interface element to the user of the first device.
 12. Thenon-transitory computer-readable storage medium of claim 11, wherein thesteps further comprise: receiving input that indicates the user'sselection of the user interface element from a plurality of userinterface elements that are selectable to select different displaysincluding the first and second displays; and in response to receivingthe input, sending, over the network, to the second device, secondinformation indicating that the user has selected the first display andnot the second display.
 13. The non-transitory computer-readable storagemedium of claim 11, wherein the steps further comprise: receiving inputthat indicates the user's selection of the user interface element from aplurality of user interface elements; in response to receiving theinput, sending, over the network, to the second device, secondinformation indicating that the user has selected a particular displaythat corresponds to the user interface element; and receiving, over thenetwork, from the second device, third information (1) that representsat least some content that is currently being displayed on theparticular display and (2) that the second device sent in response toreceiving the second information.
 14. The non-transitorycomputer-readable storage medium of claim 11, wherein the firstinformation indicates physical dimensions of the first and seconddisplays, and wherein the step of generating the user interface elementcomprises generating a user interface element that contains rectangleshaving sizes that correspond to the physical dimensions.
 15. Thenon-transitory computer-readable storage medium of claim 11, wherein thestep of generating the user interface element comprises generating auser interface element that (1) includes the word “left” and (2)indicates that contents from a leftmost display of the first and seconddisplays, rather than contents from a rightmost display of the first andsecond display, is currently being presented to the user on a thirddisplay that is separate from the first and second displays.
 16. Anon-transitory computer-readable storage medium storing instructionswhich, when executed by one or more processors, cause the one or moreprocessors to perform steps comprising: displaying a set of icons thatinclude at least (1) a first icon that depicts a highlighted rectangleto the left of a non-highlighted rectangle and (2) a second icon thatdepicts a non-highlighted rectangle to the left of a highlightedrectangle; receiving first input indicating a user's selection of thefirst icon; and in response to receiving the first input, displaying, ona local display, an image that a remote device is currently directing toa first remote display that is physically located to the left of asecond remote display.
 17. The non-transitory computer-readable storagemedium of claim 16, wherein the steps further comprise: receiving secondinput indicating the user's selection of the second icon; and inresponse to receiving the second input, ceasing to display the image onthe local display and further displaying, on the local display, an imagethat the remote device is currently directing to the second remotedisplay.
 18. The non-transitory computer-readable storage medium ofclaim 16, wherein the set of icons further includes at least a thirdicon that depicts a highlighted rectangle to the left of a highlightedrectangle and wherein the steps further comprise: receiving second inputindicating the user's selection of the third icon; and in response toreceiving the second input, concurrently displaying, on the localdisplay, both (a) an image that the remote device is currently directingto the first remote display and (b) an image that the remote device iscurrently directing to the second remote display.
 19. The non-transitorycomputer-readable storage medium of claim 16, wherein the steps furthercomprise: in response to receiving the first input, visuallydistinguishing the first icon from other icons in the set of icons;receiving second input indicating the user's selection of the secondicon; and in response to receiving the second input, performing stepscomprising: ceasing to visually distinguish the first icon from othericons in the set of icons, ceasing to display the image on the localdisplay, visually distinguishing the second icon from other icons in theset of icons, and displaying, on the local display, an image that theremote device is currently directing to the second remote display.
 20. Asystem comprising: network means; first display means; second displaymeans; first computing means for sending, over the network means,information that indicates at least a positional orientation of thefirst display means relative to the second display means; and secondcomputing means for receiving the information over the network means andfor presenting, responsive to receipt of the information, arepresentation indicative of the positional orientation of at least thefirst display relative to at least the second display; wherein the firstdisplay means and the second display means are communicatively connectedto the first computing means.
 21. The system of claim 20, wherein thepositional orientation includes both (a) a size and position of thefirst display means and (b) a size and position of the second displaymeans, and wherein the second computing means is for presenting agraphical indication of the size and position of the first display meansrelative to the size and position of the second display means.
 22. Thesystem of claim 20, wherein the positional orientation includes both (a)a size and position of the first display means and (b) a size andposition of the second display means, and wherein the second computingmeans is for presenting a textual indication of the size and position ofthe first display relative to the size and position of the seconddisplay.
 23. The system of claim 20, wherein the second computing meansis for presenting a set of icons, each particular icon of whichindicates sizes and positions of the first and second display meansrelative to each other, and each particular icon of which is selectableby a user of the second computing means to select, from a set of contentcomprising both contents shown on the first display means and contentsshown on the second display means, a different subset of the set ofcontent to be displayed by the second computing means.
 24. The system ofclaim 20, wherein the second computing means is for presenting a set oficons that indicate sizes and positions of the first and second displaysrelative to each other; wherein the second computing means is furtherfor detecting a user selection of a particular icon from the set oficons; wherein the second computing means is further for displaying,responsive to the detecting, at a first position that is based at leastin part on positional information indicated by the positionalorientation, a first frame having a size that is based at least in parton size information indicated by the positional orientation for thefirst display means; wherein the second computing means is further fordisplaying, responsive to the detecting, at a second position that isbased at least in part on positional information indicated by thepositional orientation, a second frame having a size that is based atleast in part on size information indicated by the positionalorientation for the second display means; wherein the second computingmeans is further for displaying contents from the first display in thefirst frame; and wherein the second computing means is further fordisplaying contents from the second display in the second frame.
 25. Thesystem of claim 20, wherein the second computing means is forpresenting, at the first device, a set of icons that indicate sizes andpositions of the first and second display means relative to each other;wherein the second computing means is further for detecting a userselection of a particular icon from the set of icons; wherein the secondcomputing means is further for ceasing to display, responsive to thedetecting, a first frame (a) having a size that is based at least inpart on size information indicated by the positional orientation for thefirst display means and (b) containing contents from the first display;wherein the second computing means is further for displaying, responsiveto the detecting, a second frame having a size that is based at least inpart on size information indicated by the positional orientation for thesecond display means; and wherein the second computing means is furtherfor displaying contents from the second display in the second frame.